XOOPS 4.0 “Xoosphere” development roadmap v1

Introduction

During the last years, a lot of people have been looking at the webdifferently, and showing the will to use more advanced tools. As such,the number of users subscribed to xoops.org have been multiplied by X.And if the XOOPS community has grown in number, it has also grown indiversity.

Last year, when we realized that the XOOPS projectwasn’t organized to handle so many people, we also had to admit thatthe XOOPS core architecture wasn't able to evolve as people wanted. Notso long ago, this program was just a php portal system, not even partlyObject-Oriented as it is today, and made in an almost monotheisticmanner. For the users, the enhancements added to XOOPS then XOOPS 2have been quite positive, but for developers this legacy has beenpulling us down. It has become harder and harder to contribute to thecore, and to fix some problems, and it was time for a new start.Several other reasons were also convincing us that, if that had tohappen, it had to be right now: PHP5 is already here, has greatfeatures, and one day we’ll have to decide to exploit them and forgetabout PHP4. Considering the re-architecure we plan will require severalmonths, it was best for us to do this switch right now. We cannot askdevelopers to change their methods like this every year.

Shaping the Xoosphere

Acrossthe years, the XOOPS core has evolved to become more than just a portalscript. Hovever as of today it cannot be seen as a real ContentManagement System neither as an applications framework, while in themeantime it is a bit of all of this. For this reason, people havebecome unable to define what was XOOPS, and it was the first problem wehad to solve before trying to go anywhere.

So we first tried todesign a global architecture that could be able to integrate all therequired components we could already imagine. The objective here wasnot to list all these components or to detail their individualimplementation, but to give an overall unrestrictive structure to ourfuture system mainly describing how its composing parts will relate toeach others, giving a precise place to each of them.

Once itsfirst layers implemented, starting from the lowest ones, this globalarchitecture will be enhanced, detailed, grow to include new classesand frameworks until we reach our final objective: have a platform thatwill allow us to run and maintain all the xoops.org web sites as wewould like them to be (while they are actually too much built aroundXOOPS and its limitations).

At first, the Xoosphere project is about a way more than it is about a particular program or features:

• Wewill take care of the way our code is built, enforcing designprinciples such as simplicity (not ease of use, which is somethingdifferent), and modularity. We will ensure every components has itsprecisely defined place in a globally envisioned, modular,architecture. We will take care of not coding things before we needthem, and ensure developers have a consistent, understandable, anddocumented platform.
• We will take care to respect and use webstandards where appropriate: XOOPS, like PHP, has to stay a gluebetween many more softwares, and standards are also done to facilitateinteroperability, not just to confuse programmers.
• We willtake care of the way we collaborate together, communicating aboutthings before they are implemented, ensuring additional people can jointhe core development as naturally as possible.
• Then, if allthis go well, we'll be able to take care of the way we can benefit fromthe flow of ideas and efforts visible on the xoops.org forums andwithin other teams of the XOOPS project.

Adequatemethodology should allow us to obtain a system able to satisfy as manyof the various expectations actual XOOPS users have manifested duringthe last two years as possible:

• Developers will have aconsistent and expectable platform to build on, with libraries to helpthem build any type of Web-based application, not just “XOOPS modules”.
• Professional users will have a flexible system that integrates itself well into their existing information architecture.
• End-userswill get an extremely powerful and customizable web building tool madeof specialized and easy to understand independent programs, withmetaphors interfaces they already know and understand for documentsmanagement tasks as well as in page/site building applications.

The operational / foundation layer

Exxos

Exxoswill be an object instantiation and runtime configuration frameworkdesigned to help building efficient, yet configurable and flexible PHPapplications. It will allow every object to be switched between several“execution modes” to allow programmers to make flexible, configurableand adaptive versions of their components on one side, while providingan optimized and more scalable version at the same time.

Exxositself will exploit this ability, and provide a powerful runtimeconfiguration layer that should allow people to configure every aspectof their system. In development mode its factories will create andconfigure objects according to user preferences, to generatepre-configured extended classes that they will be able to instantiatedirectly in production mode.

This framework will also providestandard text encoding services made available locally (text encodingbeing the key to higher level internationalization functionalities)that will minimize the differences between existing back-ends like theiconv or mbstring extensions, as well as a standardized way toimplement observers (or event handlers).

The XOOPS kernel

The kernel will be our own Exxos implementation, providing a globalfactory responsible of the instantiation of all the objects of thissystem, and taking care of the global boot and shutdown sequences. Itwill bring the following paradigms and features:

• The notion of user identity allowing us to make user-aware applications.
• Thepossibility to make localized applications, with the addition of alanguage service handling the management of strings collections and thetranslation of those strings (and that should normally support bothgettext and a provided php class as its translation engine).
• Apath handler allowing developers and users to address objects ormodules files and resources independently of their physical location(giving birth to the xoops:// virtual file system).
• The notionof site, or host identity. Just like some web servers can be configuredto support multiple sites or vhosts, the same XOOPS kernel will be ableto operate several sites (differentiated using the request URI). Thisfeature will be considered in the kernel factories, to make thepreferences management system able to specify global but alsosite-specific settings.
• The use of UTF-8 as this platformnative encoding (although it will still be configurable to use anyother internal encoding, for people who won’t need the benefits ofUnicode).
• The notion of application (or module). The kernelwill instantiate the current module object at the end of its bootsequence, and verify that the current user is allowed to access therequested script.
• The possibility to install additional“startup items“ that will be instantiated during every system startup,after the initialization of default services.
• A globallyavailable input filtering facility that developers will be able to useto clean up user-provided request variables before they use them. Itwill protect them from SQL injection or cross-site scripting and helpthem build applications that are safe from the earliest stages of theirdevelopment.

The architectural services

The lastpart of this operational layer will consists of architectural services.This layer will contain various low-level frameworks providingstandardized APIs to external services or systems. As most of theclasses it contains will be used regularly and on every page request (alot of them being tools that will be used by higher level components),they will be designed to stay simple and optimized, and to bringstandardization and modularization more than features.

XoopsDb

TheXoopsDb framework will benefit from the standardization PDO willfinally bring to PHP 5.1. The new version of our framework will bemodeled after the PDO interfaces, ensuring this extension can be useddirectly if it's available (giving out of the box support for all thePDO supported platforms: Ms-SQL, MySQL, Firebird, Oracle, PostgreSQLand SQLite), but we'll also provide a fall back MySQL driver (and wouldalso like to provide a MySQLi driver if people are available to do it).

Thisversion of XoopsDb will also be encoding-aware, and automatically takecare of text encoding conversion (using native driver support ifavailable, or the Exxos provided engine otherwise).

XoopsHttp

TheXoopsHttp frameworks will contain some of the Http specific components,like session handlers. An important point here is that we will ensureXOOPS 4 is build on the Http protocol, not against it. Throughout ourwhole development process, we will ensure XOOPS stays a Web buildingplatform that uses the most obvious standards like this protocol asadequately as possible. Not only will this allow us to imagine XOOPS asjust a part of a bigger platform or system, it will also be the reasonbehind some of our architectural choices, like the fact XOOPS4 willinternally manipulate canonical, human-readable URIs, that will staylike this if possible, or get transformed to infamous xxx.php ones inthe worst cases (as opposed to other programs that manipulate scriptlocations later transformed using “shortURL functions”).

XoopsTemplate

Smartywill be the first engine to be implemented in the XoopsTemplateframework (though differently than it was in XOOPS2 and with theaddition of custom extensions to the default engine ), but depending onthe way things go, we might add our custom template engine here,implemented with a compiler similar to the one Smarty has (but withoutsome its almost never used features, to keep something simple and morescalable than what Smarty has become).

Core services

Thecore services layer will provide primary services built above the lowerlevel architectural components, and acting as intermediary betweenapplication level MVC-oriented components.

XoopsAuth

TheAuthentication framework will provide two services: the authenticationservice (handling the whole authentication process) will be separatedfrom the authentication driver (taking care of checking credentialsvalidity), this last one becoming configurable. Several default drivers(LDAP, XoopsDb...) will be provided, but people will be able to designcustom ones to extend this framework possibilities.

XoopsUi

XoopsUiwill bring the base tools later used to build advanced XOOPS4applications user interfaces. One of the key elements for this will beUser Interface Files that developers will be able to use tocreate/describe UIs (some kind of super-templates that can instantiateobjects and provide several versions of an interface, made fordifferent display technologies: HTML, but also XUL, or whatever). Basedon XML, user interface files will be compiled in the same manner Smartytemplates are, to ensure a fast access on production sites.

XoopsTheme

XoopsThemewill encapsulate the functions of the page building service, that wrapsmodule output into a themed page. Page building will be enhanced willthe addition of an intermediary application-level template, and thepossibility will be given to objects to manipulate page-level meta data(so a widget inserted into a page can ensure its required style sheetsand script files are fetched from the page header). More power willalso be given to themes, as resources containers: people will now beable to use themes to customize any resource used to build the output(whether it is a template, style sheet, script file or image). Whenaccess to a themed resource will be requested, XoopsTheme willautomatically check if a custom version exists in the theme, and modifythe resource URI if it does. Access to these last features will beprovided with the help of custom Smarty compiler functions, to ensurethis doesn't become a scalability killer.

We are also considering the possibility of making some of this new framework features available in the 2.1/2.2 branch.

XoopsData

XoopsDatawill be designed to address some of the most important areas offunctionality in document-based web applications: object graphmanagement and persistence. The classes it will provide will leveragemodules developers from having to take care of retrieving or savingobjects properties from/to persistent stores, or from having tovalidate these properties values before they are saved to a persistentstorage. Some of its key features will be:

• A design using acompletely model (as in MVC) oriented approach: XoopsData applicationswill not manipulate data stored in databases, and made accessible viaData Access Objects, but adopt an opposite approach and manipulateDocuments and Content entities that will get saved to Persistent objectstores.
• It will transparently support distributed data stores(so the items an application get can come from different stores withoutit ever noticing this).
• It will provide a XoopsDb storagedriver, but also a file system based XML storage driver (to persistdocuments as XML files in the server file system), and allow thedevelopment of custom third-party drivers.
• Its data stores will be language aware (so people can store multiple translations of the same document)
• Eachstore will benefit from an own hierarchical organization system,allowing content to be accessed using the most understood pathmetaphor.
• Data stores will be expandable using custom metadata sets, to support the additions of ratings or download counts tocontent entities.

This is a framework where a lot of featurescould go. However, as with everything else we will first ensure we havesomething working here and decide own many of the fancy features wewill effectively want to implement in this version and which to keepfor later depending on the people contributing.

Application Services

TheApplication Services layer will contain several frameworks based on theMVC pattern, to help build document (model) oriented applications asfast and easily as possible. The ApplicationKit will bringXoopsData-based Object controllers made to create, view or edit manageddocuments. The WidgetsKit will provide several default widgets thatpeople can use to build powerful user interfaces (tool bars, enhancedform controls). While we don't think XOOPS should provide a full-blowncontrols library, at least we will have to provide a few defaultcomponents as well as some standardization here, so 3rd-partydevelopers can make their own custom XoopsUi widgets.

The Core Modules

The modules delivered with this new platform will fall into several categories:

• Systemmodules will provide several applications necessary to run and manageXOOPS sites, like the User logon module providing the system loginwindow, or the new modular management console.
• Contentmanagement modules will allow user to browse their documentscollections: a content explorer will allow navigation through knowndata stores, and the default document viewer / editor will allow theedition of resources and documents that are not managed by a specificapplication.
• The new XOOPS development tools will aim to givedevelopers access to a completely integrated development environment:they will have applications to visually edit classes or new documenttypes, and allow them to concentrate on real functionalities instead ofspending half of their time on repetitive tasks. At this really earlystage of development, we are imaginating these applications with an XULuser interface (this will depend on the possibilities offered by theXoopsUi classes, and of the volunteers we can get to work on thispoint).
• Some of the functionalities offered by the developertools will be reused to allow end-users to extend their sitesfunctionalities without the need to code anything manually (likecreating new model classes to add custom documents types), but whetherwe'll make these users use those applications directly or we will makesome of their functionalities available somewhere else is to be defined.
• Last,the old XOOPS2 core will be re-implemented as a XOOPS4 application.This will allow us to provide a maximum level of compatibility, as someof the classes XOOPS2 modules will use will effectively be the old 2.xones (while most low-level classes will be bridged, to give old modulestransparent access to some of the new X4 features).

OOO (Open source, Open design, Open development)

Ourmain objective during the development of this version, will be to makeXOOPS a real open-source project, made highly collectively. From what Isaw in recent forum posts, I guess most of the actual XOOPS users maynot realize this importance of this change: this has never been thecase in the past, the worst consequence of this being the inability ofthe XOOPS2 architecture to be easily enhanced in a way that could haveallowed us to continue its development by a more collective effort.

Thisgoal will bring a lot of changes to the traditional XOOPS developmentmethodology. This means everybody can forget about the “code privately,release when finished” way adopted by previous core development teams,and say “Hello” to early alpha core releases, or to the longly awaitedtechnical documentations. More precisely, these are the things youshould be able to see during the next weeks:

• Releases willbe made regularly, in a more adopted “release soon, release early” way.We expect to make one new release approximately every two or threeweeks.
• A first alpha release will be made really soon. Itwill mark the start of this project as something collective.Programmers cannot work together if they haven’t got something to workon. We will have something right after this.
• The sourceforgetools will be used during the first development stages. CVS of course,forums, whatever we find necessary. From the first weeks of June,existing mechanisms or expected enhancements will be described andexplained on sourceforge, even if the number of people effectivelyworking on the core is no more than one or two. The key to getsuccessful contribution is to make people aware of how to contribute,and a lot of effort will be made about this, from the beginning.
• Afterthe Catalysis release, the core developers will start building a newCore development site, that would give us tools more tailored to ourneeds or more customizable than the Sourceforge ones. From this point,people needs will become what drive the additions made to the core.
• Then,after developers, the new core will open itself to more and more peopleneeds, progressively. Each person contributing to the XOOPS project,team after team, will be taken into account and will become part of theXOOPS noosphere: liaison, documentation, modules repositories, publicrelations. During that last part, XOOPS will evolve to become the toolthese people need to collaborate and communicate. This will allow acomplete rebuilding of all the xoops.org websites, during the finalstages of the Xoosphere project.
• The XOOPS4 API will beentirely documented. Every class or component (maybe except reallyearly and experimental ones) will have at least phpdoc commentsattached to it. Interfaces freezing status will also be clearlyindicated, so developers are aware of what may be subject to change.
• Areal “XOOPS core technical reference library” will be written inparallel to the core development (not an “API reference” but a realdocumentation, for humans: with texts, explanations, samples... Not araw list of properties you can use to code, but something people willbe able to read, so they can understand ).

Schedule

The following is subject to change as development progresses.

Version/codename	Release date	Summary of major items
XOOPS 3.99 m1 Ganbarimasu	June 2005	Early preview release. First implementation of most of the low-level components, with unfrozen interfaces. Launch of the first core sub-projects (XoopsDb and XoopsAuth frameworks, registration module...)
XOOPS 3.99 m? Catalysis	3rd quarter 2005	First official document drafts. Very cold operational layer. Working core services, and first application level frameworks classes implemented. Usable (and used) by contributors as a base platform: beginning of the new core development site construction. Start of the final high-level core modules dev (content explorer, stores manager, pages builder...)
XOOPS 4.0 beta1 Reaction	Early 2006	Beta level classes and frameworks (with frozen interfaces). Early version of most of the core modules. Safely usable by developers and site builders. Start of the re-implementation of the XOOPS2 core. Launch of the new xoops.org building project between all the XOOPS teams.
XOOPS 4.0 RC1
XOOPS 4.0 Xoosphere	1st half of 2006	Stable release / Full developer documentation Full 2.1.x backward compatibility implemented Launch of the new xoops.org websites

References / inspiration

• Axioms of Web architecture / Design Issues (Tim Berners-Lee)
• The cathedral and the bazaar / Homesteading the noosphere (Eric S. Raymond)
• XOOPS2 source code (Various contributors, but mainly Kazumi Ono)
• Various specifications of the W3C
• Apache HTTP server v2.1 documentation
• RFC2616: Hypertext Transfer Protocol - HTTP/1.1
• Cocoa reference documentation

http://www.xoops.org/modules/wfchannel/index.php?pagenum=14